• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The invention relates to a method for detecting the presence and status of a traffic light on a road scene from the processing of images captured by at least one camera on a motor vehicle. According to the invention, a same traffic light is detected and tracked (S2, S4, S5) from captured images (Si) by a first front camera and from captured images (S3) by at least one video camera. viewing among a plurality of vision cameras of an embedded peripheral vision system. Application to the management of automatic stops and restarts of an autonomous vehicle according to the status of a traffic light.
    公开号:FR3024256A1
    申请号:FR1401678
    申请日:2014-07-23
    公开日:2016-01-29
    发明作者:Vanessa Picron；Benazouz Bradai；Guillaume Trehard
    申请人:Valeo Schalter und Sensoren GmbH；
    IPC主号:
    专利说明:

    [0001] The present invention generally relates to the field of motor vehicles, and more specifically to a method and an onboard system for detecting, from the processing of images captured by at least one camera, the presence and condition of traffic lights encountered on the road. It is known to equip certain motor vehicles with driving assistance systems using different sensors to detect, in particular, road marking lines, obstacles, other vehicles, or road signs, and in particular fires. tricolor. The detection and monitoring of the status of a traffic light are particularly important for so-called autonomous vehicles so that the system can anticipate braking phases when approaching a junction when a traffic light is on the point. change to red, and allow a restart of the vehicle when the same traffic light returns to green again. Some embedded systems developed by different car manufacturers are becoming more complete and able to communicate with other vehicles on the road but also with public infrastructure, such as traffic lights. These systems, known by the English name "Vehicle-to-X" or "Car-to-X" (the letter X representing either another vehicle or a public infrastructure), use a wireless communication network (for example, example GSM, or W-LAN) to exchange information with road infrastructures (lights, 25 crossings, works, etc.), and to communicate continuously and in real time via this wireless network in order to allow the driver to to be warned of dangers before even seeing them, or even to allow a real automatic piloting with deceleration, acceleration, stopping and restarting strategies no longer requiring the intervention of the driver. Such a system 30 is for example described in WO 2013/186379. These systems are nevertheless very expensive, and are not yet operational since it is necessary to previously equip all public infrastructures with means of communication.
    [0002] Other systems already in series on certain vehicles use the images captured by a front camera, preferably located at a point in the vehicle that best corresponds to what the driver sees, for example centered at the level of the windshield. the interior of the cabin. Different treatments on the captured images thus make it possible to detect traffic signs, for example limiting or ending speed limits, ground markings, or the presence of other vehicles traveling ahead or arriving in the opposite direction. These treatments are then used for various driving assistance applications, such as lane departure warning, automatic high beam and dipped beam switching, emergency braking or cruise control. driving speed. It is also known, for example from WO 2011/026584, to use this front camera to detect the status of a traffic light.
    [0003] FIG. 1a schematically illustrates an example of a road scene for a motor vehicle 1 traveling on a road 2 and arriving at an intersection 3. The vehicle 1 is provided with a front camera (not shown). In order to ensure that the system can detect the presence of the traffic lights, in particular the intersection light 4 and the intersection light 5, sufficiently far in advance, the camera used has a cone of vision 6 with a horizontal opening angle substantially equal to 40 °. Such a camera is thus able to detect the lights located about 100 meters ahead of the vehicle 1. In the example of Figure la, the two traffic lights 4 and 5 are in the "Red" state, as represented by the first black circle of the three representative circles of the three red / orange / green states of a conventional traffic light. By a suitable processing of the images captured in the cone of vision 6, the system on board the vehicle will be able to detect not only the presence of the traffic light 4, but also its state, and to control here a progressive deceleration and therefore comfortable for the occupants of the vehicle, until the total stop of the vehicle. The vehicle 1 is then found in the situation shown schematically in Figure lb. It is then noted that the traffic light 4 is no longer in the cone of vision 6 of the camera, so that the vehicle is not informed about a passage to the "green" of the traffic light. In other words, in this situation, it is impossible for the driver assistance system to know when it should command the restart of the vehicle. The object of the present invention is to overcome this limitation by proposing a solution which is always in the field of processing images captured by on-board cameras. This object is achieved according to the invention which relates to a method for detecting the presence and the state of a traffic light on a road scene from the processing of images captured by at least one camera on a vehicle. automobile, characterized in that the same traffic light is detected and tracked from images captured by a first front camera and from images captured by at least one vision camera among a plurality of vision cameras of a embedded peripheral vision system. This ensures that an appropriate decision can be made early enough depending on the status of a traffic light, in particular an advance decision to decelerate and then stop at the traffic light if the light has changed to a red light. followed by a restart decision when the traffic light turns green again. According to other possible additional features: the method preferably comprises successively a first step of detecting and monitoring the state of a first traffic light from images captured solely from said first front camera; a second step of detecting and tracking the status of a second traffic light from images captured only from said at least one vision camera; and a step of correlating the detections of the first traffic light and the second traffic light to deduce that the first traffic light and the second traffic light constitute one and the same traffic light; the second step may be triggered only when a first tricolor detected and tracked from images from the front camera is about to leave the field of view of the front camera; the second step can be triggered only when a first traffic light detected and tracked from images coming from the front camera is further in a red state. O thus advantageously avoids unnecessary processing if the fire detected by the front camera was green, and it is not necessary to stop at its height.
    [0004] The present invention also relates to a detection system for implementing the aforementioned method, comprising: a first front camera on a motor vehicle; a first module for processing the images captured by said first front camera, able to detect the presence and the state of a traffic light; a peripheral vision system comprising a plurality of vision cameras mounted on said vehicle and a display screen capable of recreating a peripheral image of the environment of the motor vehicle from the images captured by said plurality of cameras; and a second module for processing the images captured by at least one of said plurality of vision cameras, the second processing module being able to detect the presence and the state of the same traffic light as that detected by the first processing module.
    [0005] The front camera is preferably a horizontal aperture camera of the order of 40 °. It can be positioned including substantially horizontally centered at the level of the windshield inside the passenger compartment.
    [0006] Vision cameras are wide-angle cameras (typically of the order of 180 °) traditionally located in the front bumper, on the vehicle's exterior rearview mirrors and on the rear bumper. The vision cameras that will be used in the context of the invention will be the camera of the front bumper and / or one of the mirrors cameras.
    [0007] The invention and its various advantages will be better understood in view of the following description, with reference to the appended figures, in which: FIGS. 1a and 1b, already described, schematically illustrate a particular situation of a road scene showing the limitations of existing systems based on the use of a front camera; FIG. 2 diagrammatically represents the same road situation as FIG. 1b, in the case of a vehicle equipped with a detection system in accordance with the present invention; FIG. 3 gives a simplified block diagram of an exemplary method implemented by a system according to the invention for the detection of traffic lights. The present invention is based on the fact that autonomous or quasi-autonomous motor vehicles are generally intended to be equipped with other sensors than the front camera, these sensors being dedicated to other functionalities. Among these other functionalities, so-called "peripheral vision" systems are particularly known which make it possible to assist the driver during low-speed maneuvers, typically for parking assistance or maneuvering in reverse. To do this, a screen located at the dashboard in the passenger compartment of the vehicle is used to display different views, including a bird's-eye view ("bird's eye view" in English terminology). This is made possible by the use of several cameras on the vehicle, typically a camera at the front bumper, a camera at each left and right outside rearview mirror, and a rearview camera at the rear window. rear shocks. Each of these vision cameras are wide angle cameras, typically with a cone of vision of the order of 180 °. The images captured by these different cameras are processed by the peripheral vision system to recreat 360 ° a complete vision of the entourage of the vehicle. FIG. 2 illustrates the case of a motor vehicle 1 equipped with the front camera described above, preferably a color camera, and vision cameras making it possible to offer peripheral vision, in the same road scene situation as the one described above. described in Figures la and lb. In order not to overload the figure unnecessarily, the vision cameras were not represented, nor the vision cones associated with each of these cameras. Only the zone of peripheral vision 7 recreated from the images captured by the different vision cameras has been shown schematically in FIG. 2. It should be noted that in this case, the traffic light 4 is present in this zone of peripheral vision 7. According to the invention, provision is made to add to the image processing module associated with the peripheral vision system an image processing similar to that used by the image processing module associated with the traffic light detection system. to enable the detection of the traffic lights and their change of state by the two systems, according to the distance to which the vehicle 1 is in relation to these lights. FIG. 3 illustrates various steps implemented in a method for detecting traffic lights according to the invention: In particular, the front camera captures the images of the road scene located at the front of the vehicle 1, during a step S1. These images are provided to an image processing module of the traffic light detection system. Various known processes can be implemented by this module. For example, a processing may include detecting circles in the captured image, and then comparing to known patterns. Once a tricolor light has been detected, this fire is monitored, for example by using known filtering methods of Kalman filtering type (step S2). Whatever the treatment used, each traffic light present in the cone of vision 5, as well as its state (red / orange / green) is detected and its distance relative to the vehicle 1 is estimated so as to control if necessary the phases of Deceleration or stopping of the vehicle. In addition, the peripheral vision cameras also capture different images in their respective field of view (step S3). These images are also provided to an image processing module dedicated to the peripheral vision feature, for displaying views on a screen facing the driver. The processing module is also suitable, during a step S4, to perform an image processing similar to that carried out in step S2, for the detection and monitoring of the traffic lights and their state, completed preferably by distortion corrections. The results of the treatments resulting from these two steps S2 and S4 are then merged and correlated in real time (step S5), so as to ensure that the same traffic light and its state can be detected and then tracked, first at a distance large enough, of the order of 100 meters, by the system linked to the front camera, then by some of the vision cameras associated with the peripheral vision system, in particular the front vision camera and / or one or the other. other side cameras located in the mirrors. Different strategies can then be implemented: In particular, it is possible to provide that the vision cameras take the relay as part of the processing for detecting the fires only when a traffic light detected and tracked from images coming from the front camera is about to leave the field of view of the front camera. It is indeed possible, knowing the speed of movement of the vehicle, the estimated distance to which is a traffic light detected by the front camera, and the calibration parameters of the front camera, to estimate from when this camera frontal will no longer be able to see this fire. Alternatively, it can be provided that the vision cameras take over for the detection and monitoring of a traffic light only if it turned red while still in the field of view of the front camera.
    [0008] Indeed, in this case, the detection of the transition to the "Red" state will necessarily involve a deceleration phase depending on the estimated distance of the fire to allow a stop at the height of this fire, and the vision cameras must then be to detect a new passage to the green of the same light, and therefore a restart of the vehicle.
    [0009] Whatever the strategy adopted, the tracking of the same traffic light by one and / or the other of the systems is made possible by the knowledge of the intrinsic and extrinsic parameters of the various cameras, such as their position, their orientation, their opening angle, which allow to go from the image marker of each camera to the reference of the real world, and by the knowledge of the odometric parameters (speed of movement, acceleration) of the vehicle known elsewhere.
    权利要求:
    Claims (10)
    [0001]
    REVENDICATIONS1. Method for detecting the presence and the state of a traffic light on a road scene from the processing of images captured by at least one camera on a motor vehicle, characterized in that a same traffic light is detected and tracking from images captured by a first front camera and from images captured by at least one vision camera from a plurality of vision cameras of an on-board peripheral vision system.
    [0002]
    2. Detection method according to claim 1, characterized in that it comprises successively: a first step (S2) for detecting and monitoring the state of a first traffic light from images captured solely from said first front camera; a second step (S4) for detecting and monitoring the status of a second traffic light from images captured solely from said at least one vision camera; a step (S5) for correlating the detections of the first traffic light and the second traffic light to deduce that the first traffic light and the second traffic light constitute one and the same traffic light.
    [0003]
    3. Detection method according to claim 2, characterized in that the second step is triggered only when a first traffic light detected and tracked from images from the front camera is about to leave the field. vision of the front camera.
    [0004]
    4. Detection method according to claim 3, characterized in that the second step is triggered only when a first traffic light detected and tracked from images from the front camera is also in a red state.
    [0005]
    5. System for implementing the method according to any one of claims 1 to 4, comprising: a first front camera mounted on a motor vehicle; a first image processing module captured by said first front camera, able to detect the presence and the state of a traffic light; A peripheral vision system comprising a plurality of vision cameras mounted on said vehicle and a display screen able to recreate a peripheral image of the motor vehicle environment from the images captured by said plurality of cameras; and a second module for processing the images captured by at least one of the plurality of vision cameras, the second processing module being able to detect the presence and the state of the same traffic light as that detected by the first processing module. 15
    [0006]
    6. System according to claim 5, characterized in that the front camera is a horizontal aperture camera of the order of 40 °.
    [0007]
    7. System according to any one of claims 5 or 6, characterized in that the front camera is positioned substantially centered horizontally at the level of the windshield inside the passenger compartment. 20
    [0008]
    8. System according to any one of claims 5 to 7, characterized in that said at least one of the vision cameras is a wide angle camera located at the front of the vehicle or on one side of the vehicle.
    [0009]
    9. System according to claim 8, characterized in that the wide-angle camera is in the front bumper of the vehicle.
    [0010]
    10. System according to claim 8, characterized in that the wide angle camera is on an outside rearview mirror of the vehicle.
    类似技术:
    公开号 | 公开日 | 专利标题
    EP3195190B1|2020-09-02|Detecting traffic lights from images
    US10235581B2|2019-03-19|Vehicle vision system with traffic light status determination
    US10992908B2|2021-04-27|Vehicular vision system with dual processor control
    FR3049560B1|2019-08-09|METHOD AND SYSTEM FOR ASSISTING THE CHANGE OF THE TRACK TRACK FOR A MOTOR VEHICLE
    FR3068944B1|2019-07-19|METHOD AND SYSTEM FOR ASSISTING THE DRIVING OF AN AUTONOMOUS VEHICLE ON A CIRCULATION PATH HAVING OBLIGED PASSAGE AREAS
    CN104742911A|2015-07-01|Driving support system
    FR3063557A1|2018-09-07|DEVICE FOR DETERMINING THE ATTENTION STATUS OF A VEHICLE DRIVER, ONBOARD SYSTEM COMPRISING SUCH A DEVICE, AND ASSOCIATED METHOD
    EP2729328B1|2016-10-19|Driver aid device for making available to a driver, in anticipation, a synthetic image representative of a chosen zone around his vehicle
    FR3046769A1|2017-07-21|METHOD AND SYSTEM FOR ASSISTING THE CHANGE OF THE TRACK TRACK FOR A MOTOR VEHICLE
    FR3083507A1|2020-01-10|METHOD AND DEVICE FOR CONTROLLING THE TRAJECTORY AND INDICATOR OF A VEHICLE IN THE EVENT OF A RISK OF CROSSING A TRAFFIC LANE
    EP3209532B1|2021-05-12|Device for controlling the deactivation of a system for monitoring the path of a vehicle, according to the direction in which the eyes of a driver are looking
    FR3078035A1|2019-08-23|VEHICLE ENVIRONMENT DETECTION SYSTEM
    WO2018146400A1|2018-08-16|Method for detecting false positives relating to a traffic light
    EP3701416B1|2021-08-04|Assisting the driving of an automotive vehicle when approaching a speed breaker
    EP3268256B1|2019-06-19|Method and device for assisting with overtaking a vehicle in the presence of another, non-visible, oncoming vehicle
    WO2020016496A1|2020-01-23|Motor vehicle comprising a visual warning device alerting about danger
    FR3093854A1|2020-09-18|DRIVER ASSISTANCE SYSTEM
    FR3103437A1|2021-05-28|Method and device for determining the setpoint for a vehicle
    FR3077548A1|2019-08-09|DEVICE AND METHOD FOR ASSISTING THE DRIVING OF A VEHICLE CIRCULATING ON A RUNWAY
    FR3068667A1|2019-01-11|AUTOMATIC ACTIVATION METHOD OF A FUNCTION OF A MOTOR VEHICLE
    FR3085023A1|2020-02-21|METHOD FOR DETECTION AND LOCATION OF A CONNECTION BEVEL BETWEEN TWO TRAFFIC LANES
    FR3059813A1|2018-06-08|DRIVER ASSISTANCE SYSTEM FOR A MOTOR VEHICLE
    FR3081416A1|2019-11-29|METHOD FOR DETECTING A DEFECT OF A SENSOR EQUIPPED WITH A PARTIALLY OR FULLY AUTONOMOUS MOTOR VEHICLE.
    FR3069364A1|2019-01-25|METHOD AND DEVICE FOR MANAGING A DRIVING ASSISTANCE SYSTEM OF A VEHICLE AND ASSISTANCE SYSTEM THUS EQUIPPED
    FR3033298A1|2016-09-09|METHOD AND SYSTEM FOR VEHICLE PASSENGER WARNING IN THE EVENT OF OPENING A VEHICLE DOOR IN THE VICINITY OF A CIRCULATION PATH
    同族专利:
    公开号 | 公开日
    WO2016012524A1|2016-01-28|
    EP3195190A1|2017-07-26|
    FR3024256B1|2016-10-28|
    EP3195190B1|2020-09-02|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20100033571A1|2006-09-28|2010-02-11|Pioneer Corporation|Traffic information detector, traffic information detecting method, traffic information detecting program, and recording medium|
    DE102009040252A1|2009-09-04|2011-03-10|Daimler Ag|Device for detecting light signals of traffic light systems for a vehicle|
    US20130253754A1|2012-03-26|2013-09-26|Google Inc.|Robust Method for Detecting Traffic Signals and their Associated States|
    DE102012108862A1|2012-09-20|2014-05-28|Continental Teves Ag & Co. Ohg|Method for calibrating several environment sensors in a vehicle|CN108154711A|2016-12-02|2018-06-12|铃木株式会社|Drive assistance device|DE102012210069A1|2012-06-14|2013-12-19|Continental Teves Ag & Co. Ohg|Method and system for adapting a vehicle start-up behavior to a traffic signal system and use of the system|DE102018106756A1|2018-03-22|2019-09-26|Valeo Schalter Und Sensoren Gmbh|DETECTION OF OBJECTS FROM PICTURES RECORDED BY CAMERAS ON A MOTOR VEHICLE|
    FR3084631B1|2018-07-31|2021-01-08|Valeo Schalter & Sensoren Gmbh|DRIVING ASSISTANCE FOR THE LONGITUDINAL AND / OR SIDE CHECKS OF A MOTOR VEHICLE|
    US10852743B2|2018-09-07|2020-12-01|GM Global Technology Operations LLC|Multimodal multi-technique signal fusion system for autonomous vehicle|
    FR3093847B1|2019-03-13|2021-02-12|Psa Automobiles Sa|TRAINING OF A NETWORK OF NEURONS, TO ASSIST THE DRIVING OF A VEHICLE BY DETERMINATION OF DIFFICULT OBSERVABLE DELIMITATIONS|
    法律状态:
    2015-07-31| PLFP| Fee payment|Year of fee payment: 2 |
    2016-01-29| PLSC| Publication of the preliminary search report|Effective date: 20160129 |
    2016-07-29| PLFP| Fee payment|Year of fee payment: 3 |
    2017-07-31| PLFP| Fee payment|Year of fee payment: 4 |
    2018-07-27| PLFP| Fee payment|Year of fee payment: 5 |
    2019-07-31| PLFP| Fee payment|Year of fee payment: 6 |
    2020-07-31| PLFP| Fee payment|Year of fee payment: 7 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1401678A|FR3024256B1|2014-07-23|2014-07-23|DETECTION OF TRICOLORIC LIGHTS FROM IMAGES|FR1401678A| FR3024256B1|2014-07-23|2014-07-23|DETECTION OF TRICOLORIC LIGHTS FROM IMAGES|
    PCT/EP2015/066807| WO2016012524A1|2014-07-23|2015-07-22|Detecting traffic lights from images|
    EP15752933.0A| EP3195190B1|2014-07-23|2015-07-22|Detecting traffic lights from images|
    [返回顶部]